fix: correct DeepSeek V4 GB300 Mooncake offload metadata - #983
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Verified on a fresh, auto-expiring production child branch: the targeted apply succeeds, a second apply reports nothing to do, and c128/c256/c512 move to offload=on while c4 stays off. All four points retain their IDs, logs, traces, throughput and latency. The apply also enforces the already-merged cache-hit-rate corrections. GB200 rows are unchanged. After merge, production recovery will use Apply Run Overrides with run_id=32809502132, including database verification and cache invalidation/warmup. |
|
Production repair verified: https://github.com/SemiAnalysisAI/InferenceX-app/actions/runs/33788142197 completed the targeted apply, database verification, cache invalidation and cache warmup successfully. Production SQL, the public benchmark API and the rendered point navigator agree: c128/c256/c512 are DRAM/Mooncake; c4 stays off. Point IDs, trace/log references, throughput and latency are unchanged. GB200 data is unchanged. Representative point: https://inferencex.semianalysis.com/inference/agentic/440282 The automatic all-registry job still encounters the pre-existing run-31633154542 backfill conflict. This repair used the verified run-scoped workflow and did not modify that unrelated correction. |
1 participant
Correct the c128, c256 and c512 points from run 32809502132, submitted in SemiAnalysisAI/InferenceX#2665. Their worker logs show Mooncake DRAM caching, but the artifacts label them non-offloaded. Keep c4 non-offloaded.
Extend their existing audited backfills so re-ingestion preserves the correction. Retain the previously registered cache-hit-rate corrections; do not change throughput, latency, runtime settings or point IDs. GB200 already has its database correction.
Related master-YAML PR: SemiAnalysisAI/InferenceX#2817. Validation: focused override tests and typecheck pass; verify the targeted correction on an expiring production child branch before applying it through GitHub Actions.
Note
Low Risk
ETL-only audited backfill metadata for three benchmark points; performance metrics and point IDs are explicitly preserved, with tests guarding behavior.
Overview
Fixes mislabeled offload identity for DeepSeek-V4 GB300 dynamo-vllm AgentX points from run 32809502132 at concurrency 128, 256, and 512. Worker logs show Mooncake DRAM caching, but artifacts stored them as non-offloaded.
The change extends the existing GB200 cache-hit-rate backfill entries for those three points (same IDs and performance fields): on ingest/re-apply it sets
offloadModeto on, stripsallocated_cpu_dram_gb, and merges Mooncake KV metadata (kv_offloading: dram, backendmooncake, version0.3.11.post1) while keeping borrowed prefix-cache hit rates and throughput/latency unchanged. c4 in the same run stays non-offloaded—noset.offloadModepatch.Adds parameterized tests that assert the correction, idempotency, attempt/fingerprint matching, and that the c4 registry entry still has no offload override.
Reviewed by Cursor Bugbot for commit bde49ae. Bugbot is set up for automated code reviews on this repo. Configure here.